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Abstract 

This  paper  examines  three  different  methods  for  collecting  GIS  attribute -value  data  using  digital  pen  and 
paper  maps  and  forms.  Users  create  features  on  a  digital  paper  map  by  ticking  the  legend  entry  for  that 
feature  on  the  map  with  a  digital  pen,  and  then  marking  the  location  of  that  feature  on  the  map.  In  order  to 
enter  attribute -value  data  to  that  feature,  users  would  either:  1)  write  the  attribute -value  data  directly  on 
the  map,  2)  write  a  linking  identifier  on  the  map,  and  then  write  that  same  identifier  on  a  digital  paper  in  a 
special  — InkID”  column  of  an  Excel  form  that  has  been  exported  from  the  GIS  system4  s  database,  or  3) 
write  the  attribute  values  in  the  table  immediately  after  having  created  (or  selected)  the  map  feature  (called 
the  temporal  association  method).  Among  the  measures  examined  were  training  time  for  each  method, 
time  for  entry  of  features,  procedural  errors,  handwriting  recognition  errors,  and  user  preference.  For 
these  metrics,  temporal  association  was  fastest,  had  fewest  procedural  errors,  and  was  preferred  by  most 
field  users.  However,  the  impacts  of  these  different  workflows  on  -back  office”  GIS  personnel  was  not 
considered. 


1.  Introduction 

A  geographic  information  system  (GIS)  stores  and  displays  spatial  information,  accepts  inputs  into  a 
geographic  database  (-geo-database”)  underlying  that  geospatial  display,  and  supports  calculations  based 
on  the  geographic  information  to  support  decision-making. 

Unfortunately,  the  complexity  of  GIS  software  and  end-user  computer  equipment  has  made  it  difficult  for 
field  workers,  untrained  in  the  intricacies  of  GIS,  to  collect  needed  data.  We  have  addressed  this  need  by 
developing  digital  paper  and  pen  mapping  software  based  on  Anoto  technology  that  integrates  with  leading 
GIS  software  from  Esri  (ArcGIS),  and  TerraGo  (GeoPDF).  This  technology  enables  untrained  users  to 
provide  GIS  data  by  marking  on  digital  paper  maps  using  digital  pens.  Such  data  is  immediately  geo- 
registered  by  the  software  and  made  available  for  subsequent  processing. 

Because  data  collection  is  via  paper  and  pen,  a  broad  class  of  end  users  already  knows  how  to  use  the 
device.  For  example,  sixth-grade  students  have  used  the  technology  during  Earth  Day  events  in  Nashua, 
NH  to  indicate  on  digital  paper  maps  the  location  of  garbage  in  their  city  that  needed  to  be  picked  up. 

When  the  data  was  imported  into  the  city's  GIS  system,  GPS-equipped  garbage  trucks  could  then  be  routed 
to  the  specified  locations,  (http://www.adapx.com/sites/default/files/file/capturx-casestudy-nashua-earth- 
day.pdf) 

Although  map  markup  has  many  uses,  many  GIS  workflows  involve  the  users4  providing  attribute-value 
data  for  newly  created  or  existing  map  features.  For  example,  in  the  past,  the  US  Census  Bureau  often 
incorporated  an  -address  canvassing”  step  in  the  decennial  census,  in  which  a  census  worker  would  mark 
the  position  of  a  new  housing  structure  on  a  map,  write  a  — mp  spot”  label,  and  then  on  a  subsequent  page, 
would  enter  that  map  spot  label  into  a  table  and  provide  various  attributes  about  the  related  housing 
structure.  For  example,  the  worker  could  indicate  that  it  had  4  bedrooms,  two  stories,  was  unoccupied,  etc. 
Many  industries  that  capture  geospatial  information  have  very  similar  attribute -value  data  collection  needs. 

To  address  the  need  for  collecting  data  via  field  workers  untrained  in  the  intricacies  of  GIS,  we  have 
employed  the  use  of  digital  pen  and  paper.  This  enables  the  use  of  existing  paper-based  work  practices, 
which  minimizes  training,  while  providing  a  digital  output  that  can  lead  to  substantial  productivity  gains. 
The  question  to  be  addressed  in  this  paper  is  how  to  support  attribute-value  data  collection  via  digital  pen 
and  paper. 


For  public  release,  distribution  unlimited. 


3 


The  paper  is  organized  as  follows: 

In  section  2,  we  describe  the  background  technologies  to  be  used  -  digital  paper  and  pen,  as  well  as 
Capture™  software  that  integrates  this  technology  with  GIS  and  forms  software.  In  section  3,  we  discuss 
various  methods  that  have  been  implemented  for  capturing  attribute  value  data.  Section  4  presents  the 
study  we  undertook  to  investigate  the  advantages  and  disadvantages  of  each  method.  Section  5  discusses 
related  work.  Section  6  then  discusses  the  results  and  their  implications,  followed  by  the  conclusions  we 
draw  in  Section  7.  The  appendices  provide  the  study  materials  and  protocol. 


2.  Background  Technology 

In  this  section,  we  describe  the  basic  digital  pen  and  paper  technology  and  how  we  have  integrated  it  with 
standard  GIS  and  spreadsheet  systems  in  order  to  bridge  the  analogue/digital  gap. 


2.1  Digital  Pen  and  Paper 

Digital  pen  and  paper  technology  provides  a  dot  pattern  that  is  printed  on  a  paper  document  (see  Figure  1), 
and  a  digital  pen  that  senses  that  pattern  (see  Figure  2).  Unlike  relative -positioning  digital  pens,  these 
digital  pens  read  not  only  an  absolute  position  on  a  page;  the  pens  know  precisely  what  page  it  is.  The  pen 
records  the  coordinates  of  the  pen  on  the  page,  as  well  as  certain  meta-data  (e.g.4s  the  page 4  s  unique 
identifier,  date,  time,  etc.).  The  X-Y  data  is  stored  in  the  pen  until  it  is  unloaded  or  transmitted  to  another 
system.  Application  software  then  interprets  the  digital  ink  markings  in  the  context  of  the  specific  piece  of 
paper  they  were  written  on.  The  most  successful  digital  pen/paper  technology  is  by  Anoto,  AB 


Figure  1.  Anoto  Dot  Pattern 


Figure  2.  Digital  pen 


(http  ://www.  anoto  .com) . 


Various  software  companies  have  developed  a  number  of  applications  that  integrate  this  base  technology 
into  commercially  viable  offerings.  Most  of  those  applications  enable  the  user  to  fill  out  forms,  though  the 
design  of  the  form  is  usually  done  by  the  software  vendor.  Unique  among  these  software  applications  are 
those  from  Adapx1,  whose  Capture2  line  of  software  enables  the  end  user  to  develop  their  own  forms, 
maps,  etc.,  by  integrating  digital  pen  and  paper  technology  with  popular  software  applications,  such  as 
Esri4s  ArcGIS  and  Microsoft4  s  Excel,  OneNote,  and  SharePoint.  In  the  study  described  herein,  two  of 
these  products  are  used  -  Capture  for  Esri  ArcGIS™,  and  Capture  Forms  for  Microsoft  Excel™. 


2.1  Capturx  for  ArcGIS 

Many  users  who  wish  to  collect  GIS-related  data  in  the  field  find  that  existing  mobile  computer  systems 
can  be  too  small,  too  complex,  too  expensive,  and/or  too  fragile  to  be  used  effectively  by  their  mobile  work 
force.  Typically,  their  field  data  has  been  collected  via  paper  and  pen,  and  then  rekeyed  by  the  GIS 


Pronounced  as  "Adapts" 

2  Pronounced  as  "Capturz" 
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department  or  even  by  the  end  user  into  an  enterprise  database.  We  built  Capturx  for  ArcGIS  to  support 
precisely  these  workflows  and  user  base. 

Capturx  for  ArcGIS  enables  a  user  of  Esri4s  ArcGIS  to  create  —dature -class”  data  in  the  field  without  the 
user  having  to  manipulate  the  underlying  application,  or  to  know  anything  about  GIS.  Figure  3  shows  an 
image  of  an  ArcGIS  map  for  wildland  firefighting.  It  consists  of  a  legend  describing  a  set  of  symbols  that 
may  appear  on  the  map,  along  with  the  map  itself.  The  symbols  are  point,  line,  and  area-based. 


Capturx  is  a  plug-in  to  ArcGIS,  and  thus  operates  transparently  with  it.  With  the  plug-in,  a  user  is  able  to 
print  a  digital  paper  version  of  this  map,  along  with  the  legend,  which  then  becomes  a  user  interface.  The 
user  would  add  feature-class  instances  of  the  elements  in  the  paper4  s  legend  by  ticking  them  with  the  digital 
pen,  and  then  drawing  their  geospatial  extent.  Thus,  the  user  could  position  firefighting  information  (e.g., 
proposed  bulldozer  lines,  aerial  foam  drops,  escape  routes,  etc.)  by  touching  their  legend  items  and  drawing 
them  on  the  map.  When  the  digital  pen  is  docked,  or  the  data  is  sent  remotely  via  a  Bluetooth  connection 
and  a  mobile  phone  to  a  computer  running  ArcGIS,  the  geo-database  updates  are  incorporated  and  made 
available  digitally  for  validation. 
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Figure  3.  Capturx  for  ArcGIS  wildland  firefighting  map 

Now,  the  creation  of  features  is  only  part  of  the  field  GIS  workflow.  Users  typically  also  need  to  enter  or 
update  the  values  for  attributes  of  those  features.  Within  ArcGIS,  each  feature  classes4  attributes  and 
values  are  kept  in  a  set  of  database  tables.  In  order  to  update  them,  the  user  needs  to  provide  tabular  input 
in  addition  to  geospatial  input.  However,  ArcGIS  (version  9.3.1)  does  not  provide  an  easy-to-use 
interface  for  tabular  input  suitable  for  field  personnel.  Often,  Microsoft  Excel  forms  are  the  field  data 
collection  tool  of  choice,  and  Excel  can  be  used  as  a  front-end  to  databases.  We  thus  support  this  existing 
workflow  using  a  second  digital  pen-based  Capturx  software  tool. 


2.2  Capturx  Forms  for  Microsoft  Excel 

Capturx  software  is  integrated  directly  with  Microsoft  Office  Excel  2007,  using  the  standard  Microsoft 
ribbon,  enabling  a  user  of  Excel  to  create  a  form,  to  specify  field  content  types,  and  to  create  the  named 
range  labels  needed  to  export  the  data  from  Excel.  When  the  form  is  printed  using  the  PRINT  WITH 
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CAPTURX  button,  a  digital  =watermark‘  of  background  dots  is  created  under  the  form  contents.  Each  field 
is  given  a  datatype  (e.g.,  date,  time,  free  text,  checkbox,  text  restricted  to  a  specific  list  of  values,  etc.)  that 
provides  constraints  to  the  handwriting  recognizer.  When  the  user  fills  out  the  form  with  the  digital  pen, 
the  digital  ink  is  first  captured  (Figure  4a),  and  then  is  converted  to  text  (Figure  4b)  according  to  the 
datatype  restrictions  specified  in  the  form  cell.  Not  all  fields  need  be  converted  to  text,  such  as  a  signature 
field,  or  a  marking  on  a  body  diagram.  The  user  can  alternate  between  seeing  the  digital  ink  and  seeing  the 
converted  text  with  the  INK  button.  Thus,  the  product  stores  both  the  original  ink  and  the  converted 
results,  which  can  be  corrected  using  the  Capturx  Review  Pane.  Because  the  software  knows  precisely 
which  form  the  pen  is  writing  on,  the  user  can  move  quickly  among  documents  without  having  to  keep 
track  of  the  source  document  and  application. 


Figure  4a.  Filled  out  attribute  form  with  digital  ink  capture 
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Figure  4b.  Filled  LinkID  form  with  digital  ink  in  Figure  4a  converted  to  text 


2.3  Three  Methods  for  Solving  the  Attributes  Problem 

We  investigated  three  methods  for  collecting  attribute -value  (A-V)  data. 

1.  On-Map.  Write  attribute -value  data  on  the  map  itself.  Thus,  a  user  could  write  -Diameter=30in” 
on  the  map  next  to  or  associated  with  a  new  or  existing  feature  on  the  map. 

2.  LinkID.  A  second  general  method  takes  advantage  of  the  ArcGIS  capability  for  joining  external 
database  relations  with  relations  in  the  geo-database.  Fortunately,  Capturx  for  Microsoft  Excel 
enables  users  to  fill  out  forms  in  Excel  that  populate  a  database  relation.  In  order  to  provide  the 
needed  attribute-value  update  capability,  we  implemented  a  method  to  export  an  attribute  table 
from  ArcGIS  as  a  structure  that  is  readable  by  Excel,  with  an  automatic  population  of  datatypes 
and  handwriting  grammars  from  the  table  to  the  Excel  spreadsheet.  The  Excel  table  is  also 
provided  with  metadata  that  will  inform  ArcGIS  about  the  datatypes  when  the  data  is  returned. 
Then,  using  Capturx  for  Microsoft  Excel,  a  user  can  print  such  a  form  digitally,  fill  it  out  with  a 
digital  pen  in  the  field,  and  then  import  the  data  back  into  Excel.  From  there,  the  handwriting 
recognition  can  be  validated  and  the  form  then  imported  into  the  attribute  table(s)  in  ArcGIS,  to  be 
coordinated  with  the  map  update  (creation  of  features  on  the  map).  Since  there  may  be  more  than 
one  set  of  attributes  and  values  for  any  given  feature  class,  the  attribute -value  update  method 
needs  to  enable  the  user  to  indicate  which  feature  goes  with  which  attribute -value  data.  One 
method  for  doing  so  enables  the  user  to  associate  feature  with  the  attribute -values  by  writing  a 
linking  identifier  (LinkID)  on  the  map,  and  entering  that  same  identifier  into  an  ID  column  in  the 
relevant  A-V  table.  Joining  the  Excel  table  with  the  GIS  database  then  provides  the  association. 

3.  Timing.  Associate  the  ArcGIS  feature  with  the  most  recently  selected/touched/entered  row  in  the 
attribute  table  in  Capturx  for  Excel  attribute  table  entries.  Thus,  this  method  uses  the  timing  of 
map  and  table  update.  The  remainder  of  the  method  is  the  same  as  LinkID. 
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In  sum,  the  present  study  compares  these  three  methods  for  performing  attribute -value  updates  - 
writing  on  the  map,  using  a  linking  identifier  across  form  and  map,  and  using  timing  information 
between  map  and  form  update. 

In  the  next  section,  we  describe  each  of  these  methods  in  greater  detail. 

2.4  Specific  Attribute  Collection  Methods 

The  overall  workflow  has  four  high-level  steps: 

•  Prepare  the  digital  paper  data  collection  products 

•  Update  the  map  and/or  capture  the  attribute-value  data  in  the  field 

•  Validate  the  data 

•  Import  the  data  back  into  the  GIS  system,  using  one  or  both  of  Capturx  for  ArcGIS  and  Capturx 
for  Microsoft  Excel. 

The  details  of  this  workflow  depend  on  which  attribute -value  collection  method(s)  are  used. 

2.4.1  Map-based  Attribute- Value  Update 

The  Capturx  for  ArcGIS  software  enables  the  user  to  set  arbitrary  attributes  on  both  newly-created  and 
existing  features.  The  user  does  this  by  interacting  first  with  a  Capturx  Attributes  -action  palette”  layer  that 
has  been  added  to  the  legend  on  the  digital  paper  map  (see  figures  5  and  6).  The  interaction  method  is 
similar  to  the  standard  map  feature-class  update  method.  For  a  new  feature,  the  user  would  tick  that 
feature  class  entry  on  the  legend,  such  as  — ranhole”  or  — tility  pole”,  and  mark  on  the  map  where  that 
feature  should  be  placed.  The  user  would  add  attribute -value  data  by  next  selecting  the  Set  Attribute 
legend  entry  (see  Figure  6)  and  writing  the  attribute -value  phrase  on  the  map.  Note  that  the  phrases  can  be 
abbreviated  (e.g.,  -E)iam@36”  or  -Col@yellow”).  The  Capturx  for  ArcGIS  software  uses  the  Microsoft 
Handwriting  engine  and  various  heuristics  (such  as  to  handle  abbreviations)  to  recognize  the  handwritten 
attribute-value  ink. 

In  order  to  update  an  existing  featured  attribute  value,  the  user  ticks  the  Circle  feature  legend  entry  with 
the  digital  pen,  which  invokes  that  function  on  the  subsequent  ink  stroke.  This  function  allows  the  user  to 
circle  a  point  feature  that  already  exists  on  the  map.  Once  that  feature  has  been  selected,  the  user  would 
employ  the  Set  Attribute  legend  entry  to  provide  the  relevant  attribute -value  data  on  the  map  in  the  form 
specified  below.  If  the  user  wishes  to  supply  attribute -value  data  for  a  linear  or  area  feature,  s/he  would 

tick  the  Intersect  Feature  legend  entry,  and  then  draw  a  line  or  tick  mark  on  the  map  crossing  the  line 
representing  the  linear  or  area  feature.  Then  the  SET  ATTRIBUTE  entry  could  be  used  as  above. 
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Figure  5.  Image  of  digital  paper  map  with  attribute  collection  legend  entries 


2.4.1. 1  Example  User  Session  for  Map-based  Attribute-Value  Update 
Below  is  a  sample  list  of  user  actions  for  updating  attribute -values  on 
the  map. 

1 .  Coming  across  a  manhole  that  does  not  appear  on  the  map, 
the  user  ticks  Manhole  on  the  legend,  and  ticks  or  draws  on 
the  map  at  the  proper  location  to  create  the  feature. 

2.  The  user  ticks  Set  Attribute  on  the  legend,  and  writes 

- Dia@36 ”  on  the  map.3  (The  shorthand  -dia”  could  either 
exactly  match  the  Esri  field  alias  -dia”,  or  be  a  fuzzy  match 
for  -diameter”.) 


Urban  Recon  Attributes 

T 

Utility  Poles 

$ 

Manholes 

B 

Set  Attribute 

Q 

Circle  Feature 

% 

Intersect  Feature 

Q 

Link  to  Form 

Figure  6.  Attribute  collection  legend 


3.  The  user  ticks  Set  Attribute  on  the  legend,  and  writes  -Utility @PSE”. 

4.  The  user  ticks  Set  Attribute  on  the  legend,  and  writes  -4:ag@465-29A” . 

5.  Later,  upon  coming  across  an  existing  manhole  that  needs  attention,  the  user  ticks 
Select  Feature  and  then  circles  that  existing  manhole  on  the  map. 


3  Pilot  testing  revealed  that  various  separator  characters,  such  as  — ”  or  were  often  misrecognized  by  the  Microsoft 

handwriting  recognition  software.  It  was  found  that  the  -@”  symbol  provided  excellent  recognition  as  a  single  stroke 
character. 
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6.  The  user  ticks  Set  Attribute ,  and  writes  -Gover@Missing” . 

Note  that  the  user  does  not  need  to  know  that  blissing”  will  be  stored  in  the  database 
as  the  number  3. 

Figure  5  shows  a  map  attribute  update  -session”,  in  which  the  user  added  10  features  and  updated  6 
existing  features  by  writing  the  attribute -value  data  on  the  map. 

When  the  pen  is  docked,  the  user  will  be  able  to  see  from  the  ArcGIS  map  that  a  feature  has  been  created, 
and  by  inspecting  the  attribute  table,  will  be  able  to  see  that  the  new  feature  has  three  new  attribute-values, 
and  that  the  existing  feature  now  indicates  the  missing  cover. 

The  Microsoft  handwriting  recognizer  produces  a  set  of  ranked  handwriting  recognition  candidates  for  each 
word.  Currently,  when  writing  attribute -value  data  on  the  map  itself,  the  recognized  values  are  not 
integrated  with  a  handwriting  recognition  correction  interface,  so  the  top  recognition  candidate  as 
interpreted  by  the  software  is  chosen.  However,  the  user  may  edit/correct  these  values  using  the  writing  on 
the  map  along  with  the  Attribute  Table  in  Esri‘s  ArcGIS  or  via  the  Identify  window  accessible  by  right- 
clicking  the  created  or  edited  map  feature,  while  inside  of  an  edit-session. 


2.4.2  Attribute- Value  Update  via  Linked  Forms 

Our  prototype  software  supports  two  additional  mechanisms  to  relate  Map  features  and 
attribute-value  data  captured  in  a  form: 

(1)  Use  of  a  -kink  ID”  that  is  written  both  on  the  map  and  the  form 

(2)  Use  of  timing  information  from  the  user 4  s  ink  strokes  on  the  map  and  form,  associating 
actions  that  occur  close  in  time  (consecutively). 

To  use  form  linking,  there  are  three  steps: 


1)  Creating  the  Excel  form  from  the  attribute  table  in  ArcGIS, 

2)  Linking  the  feature  on  the  map  to  the  attribute  data  in  the  form 

3)  Importing  the  form  back  into  ArcGIS. 


1).  Two  form  manipulation  buttons  are  added  to  the  ArcGIS  command  buttons.  The  user  selects  the 
feature  class  type  in  the  table  of  contents  and  then  pushes  the  ”+”  button  displayed  at  the  right .  This 
creates  a  comma-delimited  CSV  file  for  the  attribute  table  for  that  feature  class,  which  can  be 
imported  into  an  Excel  spreadsheet.  It  also  creates  the  basic  data  types  for  each  column  by 
inspecting  the  ArcGIS  attribute  table.  In  addition,  it  also  creates  metadata  that  tells  ArcGIS 
how  to  import  the  data  back  from  Excel,  when  the  user  pushes  the  import  button  seen  at  the 
right.  The  user  can  then  modify  the  table  and  its  layout  within  Capturx  for  Microsoft  Excel, 
adjusting  field  datatypes,  grammars,  and  formatting.  Each  feature  class  has  its  own  table,  and 
thus  its  own  form. 


2).  In  order  to  add  a  value  to  an  attribute  of  a  feature  on  the  map,  the  user  employs  another  legend  item  in 
the  Attributes  Collection  legend  called  Link  to  Form  (see  Figure  6).  After  selecting  an  existing  feature, 
or  creating  a  new  feature,  the  user  ticks  the  Link  to  Form  entry  and  then  turns  to  the  appropriate  form  page 
for  that  feature  class,  writing  the  same  linking  ID  on  the  form  as  on  the  map,  then  filling  out  the  rest  of  the 
form  The  two  form-map  linkages  that  will  be  employed  in  the  study  are  described  below. 

2.4.2. 1  Example  Attribute  Collection  Using  a  Link  ID 

Here  is  a  simple  list  of  user  actions  as  an  example  of  form-based  attribute  update  with  a  link  ID: 
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1 .  Coming  across  a  manhole  that  does  not  appear  on  the  map,  the  user  ticks  Manhole  on  the  legend, 
and  ticks  the  map  at  the  proper  location  to  create  the  feature 

2.  On  the  map  page,  the  user  ticks  Link  to  Form  on  the  legend,  and  writes  -4-”  on  the  map  next  to  the 
created  feature. 

3.  On  the  form,  the  user  locates  an  empty  row,  and  writes  -4”  in  the  form  column  of  the  worksheet 
under  the  column-heading  — Ink  ID”. 

4.  The  user  writes  -46”  in  the  cell  in  the  Diameter  column,  and  writes  -Needs  Inspection”  in  the 
Condition  column. 

2A.2.2  Example  Attribute  Collection  Using  Timing 

Here  is  a  simple  list  of  user  actions  as  an  example  of  form-based  attribute  update  without  a  Link 
ID: 

1 .  Coming  across  a  manhole  that  does  not  appear  on  the  map,  The  user  ticks  Manhole  on  the  legend, 
and  ticks  the  map  at  the  proper  location  to  create  the  feature 

2.  The  user  writes  -46”  in  the  cell  in  the  Diameter  column,  and  writes  -Needs  Inspection”  in  the 
Condition  column 

2.4.2.3  Importing  the  Data  Back  into  ArcGIS 

Once  the  attribute -value  data  has  been  captured  in  the  field,  it  needs  to  be  imported.  This  is  a  two-step 
process:  importing  first  into  Capturx  Forms  for  Microsoft  Excel,  and  then  importing  into  Capturx  for 
ArcGIS. 

a)  For  each  form  written  on  in  the  field,  the  corresponding  Excel  workbook  must  be  opened  using 
Capturx  Forms  for  Excel  and  the  ink  data  should  be  imported.  At  this  time,  the  user  should 
correct  all  recognized  attribute  data,  as  well  as  any  link  ID‘s  used.  The  corrected  form  is  then 
saved. 

b)  For  each  map  written  on  in  the  field,  the  user  should  import  the  ink  into  Capturx  for  ArgGIS 
Desktop.  Attributes  linking  ID4s  should  be  corrected  manually  in  the  open  edit  session  and  saved. 
For  each  form  written  on  in  the  field,  the  user  should  select  the  new  IMPORT  CAPTURX  FORM  tool 
and  select  the  appropriate  workbook.  All  worksheet  pages  that  correspond  to  the  current  map  will 
be  imported  in  the  geo-database  using  either  linking  ID  and/or  timing  information. 

2.4.3  Advantages  and  Disadvantages  of  Each  Attribute-Value  Method 

On  Map  advantage:  By  writing  on  the  map,  the  user  can  maintain  focus  on  one  interface, 
avoiding  the  flipping  of  pages. 

On  Map  disadvantages:  Currently,  in  the  mapping  system,  attributes  are  not  integrated  with  any 
correction  interface,  so  the  top  handwriting  recognition  candidate  is  always  used.  However,  the 
user  may  edit  using  the  Attribute  Table  or  Identify  window  inside  of  an  edit-session  at  any  point. 
The  map  may  become  cluttered,  obscuring  other  important  features.  The  user  needs  to  write 
-^attribute>@”  for  each  attribute -value  to  be  entered,  which  can  be  verbose.  There  are  ways  to 
encode  such  writing,  however  that  would  require  additional  training. 

Form-linking  advantages:  Because  the  user  is  writing  a  link  ID  on  the  map  next  to  the  feature 
and  on  the  form,  it  is  easy  for  a  human  to  determine  what  features  correspond  to  which  attribute 
row  in  the  form.  Using  timing  alone,  a  human  reader  cannot  determine  this.  In  order  to  support 
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the  updating  of  existing  features,  Link  ID's  can  be  pre-assigning  to  items  in  the  DB  and  pre¬ 
printed  in  the  form. 

Form-linking  disadvantage:  Because  the  user  is  handwriting  a  link  ID  on  the  map  next  to  the 
feature  and  on  the  form,  there  are  two  chances  for  recognition  error  that  could  lead  to  the  link  ID's 
not  matching,  or  colliding  with  a  previously  recognized  ID.  The  user  must  make  sure  at  import¬ 
time  of  the  forms  and  maps  that  all  data  is  recognized  correctly  or  corrected  by  the  user. 

Timing  advantage:  Use  of  timing  requires  no  -werb”  selection  from  the  legend  palette,  and  less 
writing  for  each  attribute  collected 

Timing  disadvantage:  The  user  may  become  distracted  and  not  write  in  the  form  after  creating 
the  feature,  may  tap  a  form  row  (thus  assigning  it)  by  mistake  on  the  way  to  the  target  row,  or  may 
not  realize  what  row  to  return  to  later  to  complete  any  missing  information 

Given  these  various  methods  with  no  apriori  choice  of  the  best  one  to  use,  a  study  was  undertaken  to 
investigate  the  strengths  and  weaknesses  of  these  three  methods  in  actual  field  use. 


3.  Study 

3.1  Materials 

Subjects  were  given  a  digital  paper  map  of  a  6  block  region  of  Seattle,  Washington,  printed  from  Capturx 
for  ArcGIS,  a  digital  pen,  and  Excel  tables  for  two  feature  types  (manholes  and  utility  poles)  for  each  of 
two  method  types  requiring  them,  printed  from  Capturx  Forms  for  Microsoft  Excel,  with  ID,  attribute,  and 
value  columns.  (See  Figures  4a,  4b,  and  5). 

Printed  on  the  map  were  legend  items  for  the  relevant  feature  types,  the  attribute -value  legend,  and 
identifiers  for  existing  features.  In  order  to  investigate  the  utility  of  the  various  methods  for  existing  map 
features,  a  table  that  included  the  appropriate  existing  features4  attribute-values  was  provided  (for  the 
LinkID  method). 

3.2  Subjects 

Twenty- four  participants  (12  male,  12  female)  were  trained  and  tested.  All  participants  rated  themselves  as 
intermediate”  users  of  technology,  implying  that  they  understand  how  most  current  technology  works  (i.e.: 
texting,  internet,  etc.),  but  lack  knowledge  of  underlying  theory.  The  ages  ranged  from  18  to  63  years, 
with  a  mean  age  of  32.8  years.  All  participants  completed  at  least  a  high  school  education,  with  the 
majority  (22  of  24)  either  enrolled  in  a  bachelor's  degree  program  or  with  a  completed  bachelor's  degree. 

3.3  Tasks 

Subjects  were  told  to  locate  50  given  objects  in  a  6-block  radius  (12-13  in  each  condition,  and  10  to  be 
located  according  to  the  user's  preference),  and  update  the  map/tables  with  their  attribute  values 
accordingly. 

3.4  Procedure 

3.4.1  Design  A  within-subject  counterbalanced  design  was  employed.  Each  subject  traversed  the  40 
features  of  the  course  in  the  same  method  order  that  they  were  trained.  After  inputting  those  40  features, 
they  were  asked  to  choose  a  method  type  and  to  complete  the  collection  of  the  remaining  10  features. 

3.4.2  Training  Each  subject  was  trained  by  the  research  assistant  following  the  script  in  Appendix  A, 
until  s/he  was  able  to  perform  3  correct  instances  of  each  method  consecutively.  The  length  of  the 
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practice  session  was  timed  for  each  method  type  and  for  an  overall  training  time  value.  The  order  of 
learning  each  method  was  counterbalanced  along  with  the  method  orders. 

3.4.3  Intervention  Protocol  The  research  assistant  (RA)  did  not  intervene  during  the  test  (course) 
portion  to  assist  the  subject  with  the  data  collection  methods.  The  measurement  of  coordinates  (via  GPS)  of 
each  feature  was  performed  by  the  RA  during  the  course  for  later  validation  purposes  (geospatial  error). 
Notes  were  also  taken  during  course  by  the  RA.  These  included:  number  of  requests  for  help,  protocol 
deviations,  relevant  subject  comments,  and  other  relevant  data/events  during  the  course  (ie:  inclement 
weather  conditions).  Intervention  was  allowed  by  the  RA  after  three  (3)  consecutive  categorical  errors  by 
the  subject.  The  intervention  was  annotated  in  the  RA  notes  as  well  as  the  corrective  action,  and  was 
reviewed  during  the  analysis. 

3.5  Measures 

1 .  Time  to  train  to  criterion  —  The  time  required  to  train  a  subject  on  how  to  use  each  method.  The 
success  criterion  adopted  for  training  a  method  was  that  the  subject  be  able  to  enter  three  correct 
attribute-values  consecutively  for  that  method.  Times  spent  training  on  procedures  unrelated  to  the 
specified  methods  were  removed,  with  the  same  designated  script  read  to  each  participant.  Overall 
training  included  not  only  the  indoor/outdoor  training  in  the  specific  method,  but  also  training  on 
the  digital  pens,  and  the  specific  study  criteria. 

2.  Time  to  enter  each  attribute-value  —  The  time  to  enter  each  attribute-value  was  calculated 
separately  for  existing  vs.  new  features.  Because  the  digital  pen  captures  the  time  of  each  stroke, 
the  research  assistant  created  an  Excel  spreadsheet  from  Capturx  for  Microsoft  Excel  (see 
Appendix  B),  which  enabled  her  to  check  a  box  when  the  subject  started  entering  a  given  feature, 
and  to  check  a  second  box  when  the  subject  finished.  The  software  automatically  calculated  the 
time  difference. 

3.  Overall  error  rates  —  Error  rates  were  calculated  per  method  type.  They  were  recorded  as  means 
across  all  of  the  subjects  and  features  recorded  based  upon  the  number  of  errors  per  possible  used 
cells.  This  error  rate  gave  an  overall  baseline  of  the  errors  for  each  method  type,  which  was 
further  broken  down  by  specific  error  and  format  types  for  each  method  (see  below). 

4.  Procedural  error  rates  —  Procedural  errors  included  forgetting  to  tick  the  SET  ATTRIBUTE  legend 
entry,  accidentally  creating  additional  features,  etc.  Procedural  errors  were  counted  based  upon  the 
number  of  features  recorded  per  method  per  subject.  They  denote  the  broadest  class  of  errors  and 
could  affect  the  total  number  of  features  recorded  by  the  subject. 

5.  Recognition  error  rates  —  Handwriting  recognition  could  create  errors,  e.g.,  recognizing  a  -2” 
where  a  -7”  was  written.  Recognition  error  rates  were  calculated  at  the  word  level  as  a 
percentage  of  attribute- value  entries.  For  each  subject  the  total  number  of  cells  per  spreadsheet 
and  the  number  of  possible  map  attributes  was  computed.  For  the  forms,  this  took  into  account  all 
possible  cells  that  could  be  filled  in  relative  to  the  specific  feature  recorded/measured.  For  writing 
on  the  map,  this  took  into  account  each  attribute -value  being  entered  on  the  map.  The  types  of 
possible  errors  included  recognition  errors  for  general  text,  for  numerical  values,  for  pre¬ 
determined  word  lists 

6.  Fill-in  errors  —  For  Excel  worksheets,  a  value  is  present  where  none  was  expected;  on  map 
sheets,  an  incorrect  load  from  the  Excel  worksheets. 

Each  error  type  was  recorded  separately  for  each  method  type.  Note  that  procedural  errors  (e.g, 
the  subject4 s  not  ticking  SET  ATTRIBUTE  on  the  map  legend  before  trying  to  update  an  attribute) 
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would  lead  to  recognition  errors  since  erroneous  data  would  be  entered  into  a  cell  and  likely 
misrecognized.  Such  instances  counted  as  one  procedural  error,  but  not  recognition  errors. 

7.  Time  to  validate  —  Validation  was  performed  by  the  research  assistant,  who  was  considered  an 
-expert”  software  user.  From  the  pilot  portion  of  the  study,  it  was  determined  that  the  subjects 
would  not  validate  their  own  results,  because  the  length  of  time  to  do  so  would  have  been 
extensive  in  training  and  actual  performance.  The  time  for  validation  included  the  time  for 
identifying  the  errors,  recording  their  existence,  and  -Aiing”  the  errors  to  the  correct  value.  In 
particular,  if  a  procedural  error  was  made,  it  could  lead  to  a  cascade  of  recognition  errors.  Thus, 
-Af  ing”  the  procedural  error  might  involve  moving  one  or  more  values  to  the  proper  field  in  the 
attribute  table  or  Excel  spreadsheet.  Notice  that  the  correction  of  recognition  errors  within  the 
Excel  spreadsheet  could  take  advantage  of  the  Capturx  Review  Pane  within  Capturx  Forms  for 
Microsoft  Excel.  However,  there  was  no  comparable  recognition  review  method  for  the  map 
system.  The  validation  time  was  recorded  with  a  3 -button  stop  watch. 

8.  Number  of  requests  for  help  —  this  measure  was  tabulated  based  on  the  number  of  requests  for 
help  per  method  type  divided  by  the  number  of  features  entered  for  that  method  type. 

9.  Geospatial  error  —  Subjects  recorded  the  location  of  their  features  on  the  map  sheet  by  ticking 
the  feature  class  entry  on  the  legend  and  then  marking  the  map  with  where  they  determined  the 
feature  corresponds.  The  research  assistant  recorded  the  location  of  each  feature  by  recording  the 
center  of  mass  coordinates  as  measured  by  a  global  positioning  system  (Trimble ‘s  Yuma  Virtual  - 
GPS,  Version  0.0.1.26;  accurate  to  1.0  ft).  The  distance  between  the  two  measurements  was 
calculated  using  The  Federal  Communications  Commission  web -calculator 

(http://www.fcc.gov/mb/audio/bickel/distance.html),  and  then  converted  from  km  to  ft.  The  map  zoom 
level  was  1:1,272. 

10.  User  preference:  Based  on  the  pilot  study  results  of  qualitative  questioning  with  20  subjects,  it 
was  decided  to  break  user  preference  into  four  specific  categories,  plus  an  -overall  preference” 
category.  The  four  categories  were:  organization,  neatness,  timeliness,  and  simplicity.  A  5-point 
Likert  scale  was  used  to  assess  the  user‘s  preference  for  each  method  type,  with  the  -overall” 
category  weighted  by  a  factor  of  four  (see  Appendix  B). 

3.6  Hypotheses 

Based  on  the  method  types  and  workflows,  the  following  hypotheses  were  made: 

Writing  a  linking  ID  in  a  form  and  on  the  map  will  be  slower  and  cause  increased  procedural 
errors,  but  will  produce  the  most  accurate  recognition  results 

Writing  the  attribute/value  information  on  the  map  will  be  faster  and  have  fewest  procedural 
errors,  but  will  be  more  crowded  and  cause  more  segmentation  and  handwriting  errors 
Temporal  association  will  be  faster  than  LinkID  because  there  are  fewer  steps 
Temporal  association  will  lead  to  procedural  errors  and  be  less  preferred  because  the  artifacts 
do  not  indicate  by  themselves  how  the  attributes  are  linked  to  the  features.4 
Subjects  will  prefer  writing  directly  on  the  maps  because  there  are  fewer  steps 

3.7  Results 

We  will  refer  to  the  methods  as: 

•  Method  type  L:  using  a  linking  identifier 


4 

This  hypothesis  was  made  before  it  was  determined  to  separate  out  the  data  collection  from  the  validation  by  having  different  people 
do  these  steps.  This  turns  out  to  be  the  more  typical  work  practice. 
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•  Method  type  T :  using  timing  information  between  map  and  form  update 

•  Method  type  M:  writing  on  the  map 

•  Method  EL  indicates  method  L  for  existing  features,  and  similarly  for  ET  and  EM 

1.  Training  time  to  criterion:  The  means  for  each  training  time  were  as  follows:  Method  L-10:51, 
Method  T-  7:06,  Method  M-15:03.  These  training  times  showed  significant  differences  among 
the  methods  (ANOVA,  F  (2,  23)  =4.14,  p<0.0001).  Each  pair  of  methods  was  compared  in  a 
secondary  analysis  (L-M,  L-T,  and  M-T),  all  of  which  proved  significantly  different  (  L-M:  F 
(1,46)=14.76,  p<0.001;  L-T:  F(l,46)=12.96,  p<0.001;  M-T:  F(l,46)=48.27,  pO.OOOOOl.)  The 
mean  training  times  per  method  can  be  seen  in  Figure  7. 

An  analysis  of  order  effects  revealed  possible  differences  in  training  times  among  the  methods 
(ANOVA,  F  (2,  23)  =  55.55,  p<0.0001).  The  only  significant  order  effect  was  among  the  training 
times  for  Method  T  when  that  method  was  trained  last,  which  was  significantly  faster  than  when  it 
was  trained  first  or  second  (F  (5,  18)  =4.17,  p<0.05). 

The  mean  time  for  overall  training  for  the  24  subjects  (summing  over  all  methods)  was 
59:51.  No  gender,  occupational,  or  age  trends  were  noticed  in  the  training  times. 


Mean  Training  Time 


Figure  7.  Overall  training  time  to  criterion. 


2.  Time  for  entire  workflow  - 

a.  Time  to  enter  a  new  map  feature  and  its  attributes:  The  mean  times  to  record  new  features  and  their 
attribute  values  for  each  method  are  as  follows:  Method  L:  48  sec.;  Method  T:  39  sec.;  Method  M: 
1:03  (See  Figure  8).  Analysis  of  variance  (ANOVA)  revealed  that  there  was  a  significant  difference  in 
time  for  data  entry  among  all  three  method  types  for  new  feature  collection  (F  (2,  69)  =18.15, 
pO.OOOl).  The  methods  were  compared  individually  in  the  ANOVA:  Method  T  was  found  to  be 
significantly  faster  than  L  (F  (1,  46)  =8.58,  p<0.01);  Method  L  was  significantly  faster  than  Method  M, 
(F  (1,  46)  =10.57,  p<0.005);  and  Method  T  was  significantly  faster  than  Method  M,  (F  (1,  46)  =32.26, 
pO.OOOl).  No  order  effects  were  found. 
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Figure  8.  Mean  time  to  create  a  new  feature. 

b  )Time  to  enter  attribute  values  for  existing  map  features:  Each  method  was  used  by  each  subject  for 
corrections  to  existing  features  at  least  twice  per  method  (i.e.,  at  least  six  times  total  per  subject).  The 
mean  times  across  all  subjects  for  performing  updating  attribute  values  for  existing  features  are  as 
follows:  Method  EL:  39  sec.;  Method  ET:  49  sec.;  Method  EM:  1:01  (See  Figure  9),  which  are 
significantly  different  when  compared  in  an  ANOVA  (F(2,69)=8.58,  p<0.0005).  More  specifically, 
attribute-value  update  with  Method  EL  was  found  to  be  significantly  faster  than  with  Method  EM 
(ANOVA,  F  (1,  46)  =15.75,  p<0.001).  The  primary  factor  that  influenced  this  difference  was  that  for 
Method  EL,  it  was  only  necessary  to  overwrite  the  data  in  the  attribute  table,  whereas  the  user  of 
Method  EM  had  to  select  the  feature  and  write  the  new  attribute/value.  No  difference  was  found 
between  Method  ET  and  the  other  two  method  types,  and  ANOVA  revealed  no  order  effects. 

Mean  times  to  update  an  existing  feature's  attributes 
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Figure  9.  Mean  time  to  update  an  existing  feature. 


3.  Overall  error  rates:  The  mean  overall  error  rates,  including  both  recognition  and  procedural  errors, 
per  method  type  were  as  follows:  Method  L:  0.032;  Method  T:  0.050;  Method  M:  0.251  (See 
Figure  10).  Compared  cumulatively  in  ANOVA,  there  were  found  to  be  significant  differences  in  the 
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error  rates  (F  (2,  15)  =12.6,  pO.OOOl),  with  Method  M  showing  significantly  more  errors  than  either 
Method  T  or  Method  L  (F  (1,  8)  =41.5,  p<0.001),  (F  (1,  8)  =40.9,  p<0.0005)  respectively).  The 
difference  in  error  rate  between  Method  L  and  Method  T,  was  also  statistically  significant 
(F(l,46)=5.68,  p<0.05). 

4.  Procedural  error  rates :  The  number  of  procedural  errors  per  method  type  was  recorded  as  an 

independent  data  set,  and  by  method  type.  Method  L  had  a  cumulative  procedural  error  rate  over  all 
subjects  of  4%  of  possible  cells  recorded.  This  largely  reflected  subjects4  not  selecting  the  Link  to  Form 
legend  entry  after  determining  a  linking  identifier  to  link  the  map  sheet  to  the  attribute  table  in  Excel, 
or  selecting  that  legend  entry  in  an  incorrect  order.  Subjects  had  no  errors  using  Method  T.  Method  M 
had  an  error  rate  irrespective  of  the  number  of  features  recorded:  Thirty-eight  extra  features  were 
marked  on  the  map  that  did  not  correspond  to  any  attribute  value  data,  as  compared  to  the  299  new 
features  recorded  using  method  M  that  did  correspond  to  attribute  value  data  (procedural  error  rate  of 
12.7%).  The  procedural  errors  for  method  M  were  primarily  the  subject's  making  a  mistake  during 
data  collection  and  subsequently  performing  the  feature  placement  again. 


Overall  Error  Rate 
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Figure  10.  Overall  Error  rates 


5.  Recognition  errors:  The  recognition  errors  across  the  different  Method  types  are  as  follows:  Method 
L:  3.2%;  Method  T:  5%;  Method  M:  25%  (See  Figure  1 1).  Method  M  showed  significantly  more 
errors  than  either  Method  T  or  Method  L.  In  addition,  Method  L  and  Method  T  were  significantly 
different  (  F  (1,  46)  =1.19,  p<0.05). 
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Figure  11.  Error  rates  across  method  types. 


Recognition  error  rates  (as  percentages)  were  compared  across  the  different  formats  of  Excel  spreadsheet 
cells  and  map  sheet  attribute  table  cells  for  three  different  types  of  recognition  errors:  numerical,  word  list, 
and  general  text.  The  percent  error  rates  for  the  Excel  spreadsheet  cells  were,  respectively,  3.0%,  1.6%, 
and  14.9%  (See  Figure  12),  which  were  found  to  be  significantly  different  (ANOVA,  F  (2,  60)  =9.45, 
p<0.001).  A  follow-up  ANOVA  was  also  run  between  of  each  pair  of  error  types.  There  was  no  significant 
difference  found  between  the  word  list  error  rates  and  the  numerical  error  rates  for  this  population  within 
the  Excel  spreadsheet  cells.  However,  the  word  list  error  rates  and  the  general  text  error  rates  were  found  to 
be  significantly  different  (ANOVA,  F  (1,  40)  =10.5,  p<0.005),  as  were  the  numerical  and  general  text  error 
rates  (F  (1,  40)  =9.29,  p<0.005). 

For  the  map  sheet  attribute  table  cells,  the  three  types  of  recognition  error  rates  were  also  calculated 
(numerical,  word  list,  and  general  text),  yielding  1 1.6%,  6.5%,  and  24.2%,  respectively  (See  Figure  12). 

An  ANOVA  was  run  among  these  three  different  types  of  recognition  error  rates  for  the  map  sheet  attribute 
table  cells  and  also  between  the  pairs  of  each  recognition  error  type.  No  significant  differences  were 
found. 


Numerical: 

#  errors/total  # 

numerical 

cells 

% 

Numerical 

Errors 

WL:  # 

errors/  total  # 

WL  cells 

%  Word 

List 

Errors 

General  text:  # 

errors/  # 

total  general 
cells 

% 

General 

text 

Errors 

Excel 

spreadsheet 

cells 

98/3401 

3.0 

21/1307 

1.6 

159/1068 

14.9 

Map  sheet 
attribute 
table  cells 

360/3108 

1F6 

119/1825 

6.5 

360/1490 

24.2 

Figure  12.  Error  rates  across  the  sheet  types  (not  the  methods). 


6.  Fill-in  errors :  No  significant  differences  across  methods  were  found  in  the  number  of  fill-in  errors. 

7.  Time  to  validate:  For  each  subject  the  total  number  of  cells  per  Excel  spreadsheet  and  the  number  of 
possible  map  sheet  attributes  was  recorded  for  a  baseline  measurement.  For  the  Excel  spreadsheets, 
this  took  into  account  all  possible  cells  that  could  be  filled  in,  relative  to  the  specific  feature 
recorded/measured.  For  the  map  sheets,  this  took  into  account  each  line  of  written  text  on  the  map 

For  public  release,  distribution  unlimited. 


18 


sheet  individually  and  how  accurately  it  filled  into  the  attribute  tables.  The  types  of  errors 
possible/observed  per  cell  included  recognition  errors  for  general  text  and  for  pre -determined  word 
lists,  and  fill-in  errors  (see  section  3.5).  Each  error  type  was  recorded  separately  for  each  method  type. 
The  mean  times  (minutes: seconds)  for  data  validation  for  the  three  different  methods  are  as  follows: 
Method  L  (1:25),  Method  T  (1:36),  Method  M  (19:23).  These  validation  times  were  considered 
significantly  different  when  all  three  times  were  compared  in  ANOVA  (F  (2,  12)  =41.03,  p<0.0001). 
While  Methods  L  and  T  were  clearly  different  from  Method  M,  Method  L  and  Method  T  were  not 
found  to  be  significantly  different  when  compared  alone  with  respect  to  validation  times. 

8.  Number  of  requests  for  help:  Of  all  of  the  requests  for  assistance,  the  majority  were  for  Method  L  (33), 
with  five  assistance  requests  made  for  Method  T,  and  1 1  requests  made  for  Method  M.  The  rates  per 
method  per  feature  created  or  updated  are  as  follows:  Method  L:  10%;  Method  T:  1%,  and 
Method  M:  3%.  The  assistance  requests  made  were  primarily  appeals  for  workflow  reminders.  For 
method  F,  the  assistance  requests  made  related  to  subjects4  not  remembering  the  proper  order  in  which 
to  link  the  map  sheet  to  the  Excel  spreadsheet(touch  LINK  TO  FORM,  assign  a  number  on  the  map  sheet, 
assign  the  same  number  on  the  Excel  spreadsheet). 

9.  Geospatial  error:  Of  the  24  subjects,  21  subjects  showed  consistent  placement  of  features,  with  a 
mean  distance  of  7.3  feet  with  a  standard  deviation  of  7.1  feet  (showing  most  estimated  locations  were 
between  a  distance  of  0  feet  and  15  feet  from  the  -actual”  location).  This  represented  approximately 
1048  features.  Three  subjects4  geospatial  data  was  left  out,  because  of  excessive  error,  such  as 
placement  of  features  on  the  wrong  street  or  the  wrong  side  of  the  street.  These  subjects  were  referred 
to  the  intervention  protocol  during  the  testing.  However,  their  geospatial  data  were  considered  as 
outliers.  The  map  zoom  level  of  1:1,272  inches  or  1”  =  1064,  so  a  mean  accuracy  of  7.3  feet  means  the 
subjects  positioned  entities  accurately  within  0.069  inches  on  the  page. 

10.  User  Preference:  Overall,  the  stated  subject  user  preference  was  Method  L:  3;  Method  T:  17; 
Method  M:  4.  While  Method  T  was  the  clearly  preferred  method,  the  actual  demonstration  of  user 
preference  (based  on  which  method  the  subject  chose  to  use  for  the  last  10  features)  was  as  follows: 
2(E),  16(T),  6(M).  Most  subjects  (but  not  all)  were  aware  of  what  they  in  fact  preferred. 

1 1 .  Miscellaneous:  It  was  noted  that  inclement  weather  may  have  affected  handwriting  clarity  (neatness), 
particularly  if  gloves  were  worn  during  the  data  collection.  Of  the  five  subjects  who  wore  gloves 
during  data  recording,  significantly  higher  error  rates  in  handwriting  recognition  were  found  (ANOVA, 
F  (5,  12)  =0.44,  p<0.05).  However,  in  determining  specific  patterns  for  why  this  was  so,  no 
significant  difference  between  the  method  types  could  be  discerned.  In  fact,  two  subjects,  neither  of 
whom  wore  gloves,  provided  the  highest  number  of  errors  due  to  poor  handwriting  and  a  slightly 
skewed  orientation).  No  particular  trend  was  noticed  between  genders  or  occupation  with  respect  to 
who  wore  gloves  or  had  excessively  bad  handwriting.  Overall,  no  particular  trend  was  noticed  between 
genders  or  occupations  for  this  study. 


4.  Discussion 

Figure  13  summarizes  the  findings  of  the  study.  Typically,  lower  values  are  better  than  higher.  Statistical 
significance  of  each  finding  is  listed  in  the  last  column.  In  review  of  the  expected  versus  the  observed 
results,  it  was  found  that  not  all  of  the  originally-postulated  hypotheses  held. 

-  Speed  of  attribute-value  data  entry:  It  was  originally  hypothesized  that  Method  F  would  be  the  slowest  in 
recording  new  features,  while  Method  M  would  be  the  fastest.  However  it  was  observed  during  the  testing 
that  Method  F  was  neither  the  slowest  nor  the  fastest,  and  Method  M  was  actually  the  slowest.  This  is  most 
likely  due  to  the  necessity  of  writing  not  only  the  attribute  value  on  the  map  sheet,  but  the  attribute  type 
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itself  (i.e.,  attribute@value)  during  Method  M.  Figure  13  shows  the  speed  of  the  method  types  as  T<L<M. 
For  existing  map  features,  method  L  was  faster  since  the  user  needed  only  to  find  the  appropriate  row  in 
the  table  via  the  LinkID  and  to  update  only  the  appropriate  attribute  value.  Method  T  required  the  user  to 
rewrite  the  entire  row  of  attribute -values. 

-  Procedural  Errors:  It  was  hypothesized  that  Method  L  would  have  more  procedural  errors,  while  Method 
M  would  have  the  fewest  procedural  errors.  The  results  showed  the  opposite,  that  Method  L  had  neither  the 
most  nor  least  procedural  errors  while  Method  M  had  the  most  procedural  errors  (  12.7%)  because  of 
erroneous/accidental  creation  of  features. 

It  was  hypothesized  that  Method  T  would  lead  to  more  procedural  errors,  but  in  the  context  of  this  study,  it 
led  to  none.  In  actuality,  it  had  the  fewest  steps  for  subjects  to  track.  However,  that  particular  hypothesis 
could  be  indicative  of  the  validation  procedure  (as  a  whole  for  time)  as  well. 

Method  L  was  the  fastest  to  validate  because  the  artifacts  did  indicate  in  a  timely  manner  which  data  in  the 
Excel  spreadsheet  charts  belonged  to  which  map  sheet  point  features.  Validation  for  the  other  method  types 
was  much  more  difficult,  because  each  line  written  on  the  map  sheet  had  to  be  analyzed  carefully,  and  each 
unlabeled  point  feature  matched  with  its  corresponding  data. 

-  Recognition  errors :  Method  L  did  have  the  least,  as  originally  hypothesized.  The  recognition  error  rates 
showed  variability  within  each  method,  as  evidence  by  the  results,  and  also  across  each  format.  It  would  be 
inappropriate  to  compare  recognition  error  rates  between  the  formats  because  Capturx  for  Microsoft  Excel 
and  Capturx  for  ArcGIS  used  different  handwriting  recognition  programs.  However,  in  prior  tests,  we  have 
found  these  recognizers  to  deliver  essentially  equivalent  performance  across  lists,  numbers,  and  general 
text. 

With  respect  to  all  of  the  attribute  data  that  was  written  on  maps  for  Method  M,  it  should  be  noted  that  there 
may  have  been  skewing  of  the  data  because  of  variable  handwriting  orientation.  Although  they  were 
counseled  to  write  as  horizontally  as  possible,  subjects  may  have  had  a  more  difficult  time  writing  in  the 
correct  orientation  in  this  format  because  there  were  no  discemable  writing  guides.  Furthermore,  the 
subjects  had  to  write  the  attribute  -type”  first,  and  if  it  was  written  incorrectly  or  was  unrecognized  by  the 
recognition  program,  the  attribute -value  was  not  filled  in  appropriately.  Since  all  three  types  (word  list, 
numerical,  and  general  text)  of  recognition  data  were  written  this  way  (attribute @value),  this  may  be  the 
reason  why  no  recognition  error  differences  were  found  between  these  data  types. 

-  User  Preference:  It  was  hypothesized  that  the  subjects  would  prefer  the  Method  M  because  writing  on  the 
map  has  fewer  steps.  Overall,  the  subjects  stated  that  they  preferred  Method  T,  and  mostly  for  the  same 
reason  that  was  assumed  for  Method  M:  it  supported  the  fastest  performance.5  However,  in  bad  weather 
(windy,  cold,  rain),  subjects  preferred  Method  M  because  there  were  fewer  papers  to  manipulate. 

In  the  survey  that  was  given  after  the  subjects  performed  their  50 -feature  collection,  the  categories  of 
organization,  simplicity,  neatness,  and  timeliness  came  up  as  the  factors  that  were  important  with  respect  to 
why  they  preferred  one  method  over  the  others.  These  factors  were  not  ranked  with  respect  to  importance 
to  the  subjects,  but  they  were  given  numerical  weight.  The  discernable  patterns  that  this  data  described  were 
only  related  to  the  timeliness.  The  subjects  all  considered  Method  T  the  quickest  to  use  for  data  collection. 


5  Notice  that  is  as  an  extra  step  over  method  M,  namely  switching  to  the  form,  However,  the  extra  handwriting  likely  was  slower  than 
flipping  pages. 
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during  testing  (Most=best) 

3 

1 

2 

N/A 

User  preference  stated  after 
testing  (Most=best) 

3 

1 

2 

N/A 

Figure  13:  Comparison  of  Method  Types 


The  upshot  of  this  analysis  is  that  temporal  linkage  is  both  a  robust  (no  procedural  errors)  and  preferred 
method  for  the  field  data  collector.  However,  it  is  clear  that  the  LinkID  method  will  be  advantageous  for 
the  data  validator.  LinkIDs  are  also  advantageous  for  updating  existing  features.  By  combining  both  of 
these  methods,  a  robust  method  can  be  built  that  meets  the  concerns  of  both  stakeholders.  The  combined 
method  would  be  simplest  by  treating  temporal  linkage  as  primary,  with  the  LinkIDs  being  used  in  a 
secondary  fashion.  They  are  useful  for  the  validator  if  there  is  need  to  correct  any  information  that  is  mis¬ 
directed.  A  formal  join  is  not  necessary.  Given  that  there  are  two  opportunities  for  handwriting 
recognition  failure  with  the  LinkIDs  when  creating  new  features,  the  temporal  linkage  will  be  most  robust, 
provided  the  user  obeys  the  temporal  constraints  of  selecting  a  feature,  ticking  the  set  attribute  legend  entry, 
and  filling  out  the  form. 


5.  Related  Work 

The  standard  way  to  collect  GIS  attributes  is  with  a  handheld  device,  such  as  Esri's  ArcPAD,  Trimble's 
Nomad,  or  TerraGo  Mobile  software  running  on  a  Tablet  PC.  For  the  US  Census  2011  address 
canvassing  survey,  Harris  Corporation  built  a  special  purpose  device.  In  general,  such  mobile  devices  can 
require  significant  training,  cost,  field  service,  software  integration,  and  are  objects  of  distraction  and  theft. 
Smartphones  and  tablets  currently  are  popular  devices,  but  share  many  of  the  aforementioned  issues.  For 
these  devices,  the  user  alternates  between  map  and  form,  since  most  of  the  devices  have  minimal  screen 
territory.  However,  for  many  field  users,  especially  those  who  are  brought  into  the  data  collection  task 
quickly,  such  as  during  an  emergency,  paper  maps  and  forms  are  the  simplest  work  practice  to  train  and 
deploy. 
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A  number  of  research  efforts  have  experimented  with  digital  paper  maps,  including  RASA  (McGee  et  ah, 
2001,  2002;  NISMap  (Cohen  &  McGee,  2004),  and  Norrie  &  Signer  (2005).  However,  these  systems  did 
not  integrate  with  a  full  geographical  information  system,  and  thus  did  not  have  the  problem  of  providing 
attribute-value  data. 

There  are  a  number  of  vendors  of  digital  paper  forms  on  the  market  (e.g.,  Destiny  Wireless 
(http ://www. destinyplc .com/ flash/#/home/ overview/) ,  MiCo  (http://www.mi-corporation.com/  ),  Expedata 
http ://www. expedata.net/ industry/industry.html )) ,  virtually  all  of  which  employ  the  Anoto  forms  toolkit 
(http ://www. anoto .com ).  These  forms  tools  are  not  populated  by  a  GIS  system  to  automatically  create  an 
attribute  table,  nor  have  they  been  used  to  collect  data  that  is  returned  to  a  GIS  system. 


6.  Implications  and  Recommendations 

The  results  of  this  study  indicate  that  each  method  has  its  own  strengths  and  weaknesses.  From  the 
perspective  of  the  field  data  collector,  who  were  the  subjects  of  the  study,  the  time -based  method  was  a 
clear  winner,  supporting  the  fastest  performance  with  the  fewest  procedural  errors,  leading  to  the  highest 
user  preference.  We  initially  were  concerned  that  the  inability  for  the  GIS  data  validator  to  be  able  to  link 
the  row  of  the  attribute -value  table  with  the  feature  on  the  map  would  lead  to  confusion.  In  principle,  this 
does  still  appear  to  be  possible,  but  this  small  study,  which  used  only  one  validator,  did  not  address  the 
issue. 

The  LinkID  method  supported  the  fastest  performance  for  updating  existing  features.  Surprisingly,  though 
we  had  thought  it  would  be  the  most  reliable  method  (though  from  the  perspective  of  the  validator),  it 
turned  out  to  have  significantly  more  procedural  errors  than  the  time -based  method,  and  more  requests  for 
help.  This  was  due  to  the  fact  that  the  method  required  that  the  legend  entries  be  selected  in  the  right  order 
(first  circle  the  feature,  then  select  the  Link  To  Form  entry,  etc.).  Subjects  either  made  order  mistakes,  or 
they  requested  help  to  remember  the  proper  order.  Other  user  interface  designs  could  perhaps  avoid  these 
issues. 

Linked  and  time -based  forms  may  be  used  at  the  same  time.  Separate  linked  (with  link  column)  and  time- 
based  (without  link  column)  forms  may  be  printed  and  used  for  the  same  or  different  feature  classes  and 
used  in  any  order.  A  single  form  (containing  a  link  column)  may  be  used  for  both  linked  and  time -based 
collection.  If  the  user  forgot  to  link  cells  explicitly,  the  system  could  fall  back  on  timing.  For  example,  the 
software  could  use  valid  timings  to  assign  rows;  otherwise  it  could  fall  back  on  Link  ID ‘s  -  so  time  would 
always  be  preferred  unless  the  user  waited  over  60  seconds  to  touch  a  row  after  creating  a  feature.  Link  ID 
rows  could  also  take  precedence.  However,  this  may  be  confusing  to  both  the  user  and  the  software, 
because  some  link  cells  may  be  empty  while  others  are  filled  in,  and  there  could  be  a  conflict  between  the 
two  methods  if  the  user  creates  a  new  feature  with  a  Link  ID,  and  then  writes  a  different  Link  ID  in  a  new 
row.  One  way  to  address  this  would  be  to  have  a  new  mode  that  uses  both  Links  ID‘s  and  time,  but  uses 
each  to  reinforce  the  other  -  both  methods  must  agree,  or  the  import  process  can  flag  a  possible  error.  This 
would  give  the  advantage  of  the  map  and  form  artifacts  of  Link  ID,  but  would  flag  any  recognition  error  of 
the  Link  ID‘s  not  matching.  A  user  study  could  determine  if  such  potential  issues  arise  in  practice. 

For  configuration  and  circumstances  of  data  collection  explored  here,  with  two  or  more  attributes  to  be 
collected,  the  writing  on  the  map  method  was  inefficient.  It  tends  to  be  more  labor  intensive  not  only  for 
the  field  user  but  for  the  validator  as  well.  It  may  be  best  in  circumstances  where  only  one  attribute  per 
feature  is  to  be  collected,  or  in  bad  weather,  when  flipping  between  pages  and  formats  would  be 
unnecessary  or  undesirable,  and  the  data  matching  between  the  formats  would  be  obvious.  Further,  for 
expert  users  of  this  attribution  technique,  there  could  be  other  methods  of  encoding  the  way  the  data  is 
written  on  the  map  sheet  (e.g.,  instead  of  attribute^ value,  write  value@ value @ value  when  the  values  are 
unique  enough  that  the  related  attribute  can  be  inferred  reliably).  Either  or  both  of  the  form-based 
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approaches  can  also  be  used  with  on-map  attribute  collection,  as  described  in  Section  2.4.1.  The 
requirement  to  write  horizontally  in  landscape  orientation  is  a  temporary  one,  with  future  iterations  of  this 
software  potentially  taking  advantage  of  automatic  baseline  detection.  Still,  writing  on  a  dense  map  can 
obscure  important  information  that  may  be  needed  at  some  future  time.  Last,  without  a  correction 
experience  for  the  map  sheet  akin  to  the  correction  available  for  the  Capturx-enabled  Excel  spread  sheets, 
the  efficacy  of  this  method  type  is  further  decreased,  especially  for  the  validator. 


7.  Conclusion 

The  temporal  linkage  method  between  the  data  collection  formats  was  overall  the  fastest  and  most  preferred 
attribute-value  update  method  for  this  study. 

All  three  of  the  methods  tested  could  be  successful  in  specific  circumstances  that  would  engage  their 
strengths  during  attribute  collection,  including  the  physical  situation  of  data  collection.  The  use  of  all  three 
of  these  methods,  based  upon  the  field  user 4  s  attribute  collection  needs  at  any  given  time,  may  be  possible 
and  prove  optimal.  It  must  be  noted  however,  that  implementation  of  all  of  these  methods  in  a  combined 
technique  may  prove  more  difficult. 

Finally,  this  study  concentrated  on  the  field  data  collector.  If  it  had  considered  more  heavily  the  needs  and 
workflow  of  the  validator,  the  performance  of  and  preference  for  each  method  might  be  different. 
Ultimately,  the  preference  for  the  method  to  be  used  is  up  to  the  organization  and  the  criteria  they  wish  to 
optimize.  This  study  exposes  some  of  the  issues  that  need  to  be  considered. 
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Appendix  A:  Training  Script 

1 .  SIGN  INFORMED  CONSENT  FIRST-  Please  read  the  Informed  Consent  forms  (2),  and  I  will 
answer  any  questions  that  you  have. 

2.  RA:  sign  IC  forms  also,  determine  method  order  from  script 

3.  (To  be  read  to  subject):  The  Study  is  titled  Soldier  as  a  Sensor.  This  is  sponsored  by  the  Army 
Corps  of  Engineers.  The  purpose  of  this  study  is  to  compare  the  use  of  three  (3)  different  methods 
of  data  collection  with  the  digital  pen.  This  data  will  be  comprised  of  major  features  such  as 
manhole  covers,  and  utility  poles.  In  these  methods,  you  will  be  making  marks  on  a  map  that 
represent  an  object  and  then  filling  in  things  about  that  feature,  such  as  color,  width,  length,  and 
identifying  marks  either  directly  on  the  map  or  on  a  spreadsheet.  These  three  methods  will  be 
physically  shown  to  you  inside  during  training,  which  will  take  about  20  minutes.  Next,  you  will 
practice  outside  using  these  on  the  street,  which  is  similar  to  the  test  route.  Outside,  you  will  train 
until  you  demonstrate  proficiency,  which  is  three  features  and  their  attributes  recorded  correctly 
per  method  (at  least  9  total).  Last  you  will  perform  the  test  using  50  different  features  on  the  pre- 
designated  route.  This  is  actual  study  portion.  The  time  it  takes  for  you  to  measure/record  all  of 
the  data  will  be  timed.  Remember,  it  is  the  method  that  is  under  scrutiny,  not  you  personally.  I 
will  be  following  you  and  taking  GPS  measurements  of  the  features,  for  later  comparison  between 
the  equipment  used.  Each  point  must  be  measured,  no  -copying”  is  allowed  of  prior 
measurements. 

4.  During  the  actual  course  portion  I  am  not  allowed  to  assist  you.  After  the  course  completion,  we 
will  return  to  Adapx  and  I  will  ask  you  some  questions  about  the  methods,  training,  and  course. 
After  this,  you  will  be  released.  As  per  our  study  guidelines,  we  are  budgeted  2-3  hours  for 
completion  of  both  training  and  execution  of  the  course.  You  will  be  compensated  a  set  amount  of 
$50  for  this. 

Show  and  tell: 

5.  Digital  pen-  This  is  the  digital  pen  (penx).  This  pen  acts  as  a  recording  device  for  strokes  that  are  made  on  a  digitally- 
rendered  paper.  Charts  and  maps  are  watermarked  to  act  as  -live”  databases.  So  you  make  a  mark  on  the  watermarked 
paper,  the  pen  records  it,  and  when  you  are  finished  recording  all  of  your  data  the  pen  is  docked  into  a  station.  That  data  is 
then  uploaded  from  the  pen  by  me  into  pre-designated  databases  (after  your  part  of  the  study  is  completed).  Proper 
orientation  of  the  pen  on  the  watermarked  paper  is  important:  the  camera  has  to  -dhce”  the  paper  or  it  will  -buzz”.  Please 
only  touch  the  buttons  on  the  page,  no  excessivge  marking  is  necessary,  and  may  degrade  the  watermarked  portions  of  the 
paper,  making  your  pen  buzz  excessively.  For  the  sheets,  only  the  watermarked  portions  are  recorded.  Any  space  outside 
of  the  watermark  (dots)  is  not  recorded.  The  sheets  must  be  properly  oriented  when  writing  on  them  as  well  in  order  for  the 
test  to  be  recognized  appropriately.  East-to-west  (L-R)  is  the  proper  orientation  with  the  top  of  the  page  aligned. 

(Physically  shown  details). 

6.  The  data  that  you  fill  in  with  your  digital  pen  is  considered  -active  ink”.  These  markings  may  be  erased  by  you  (such  as 
the  eraser  on  a  pencil).  In  order  to  perform  this  function  on  the  spreadsheets: 

For  cells  that  need  correction- 

Cross  out  the  old  value  of  a  cell  with  the  digital  pen,  touch  the  eraser  button,  enter  new 
value  within  the  confines  of  the  cell. 

For  cells  that  should  be  empty- 

Touch  the  eraser  button,  touch  the  cell. 

For  map  sheets,  the  eraser  button  is  not  available  for  your  ink.  Please  perform  the  data 
collection  again  with  the  correct  method  and  correct  data. 

The  data  that  is  already  filled  in  your  spreadsheets  is  not  considered  -active  ink”.  This  is  not  ink  that  you  have  filled  in,  and  therefore 
may  be  erased/corrected  within  whichever  method  that  you  are  performing. 

7.  Some  terms  I  may  be  using:  features-  the  things  that  we  are  measuring:  manholes,  utility  poles.  There  are  two  feature 
classes:  utility  poles  and  manholes.  Existing  features  are  those  pre-programmed  into  the  cellx  and  mapx.  Attributes-  the 
stuff  we  are  measuring  or  recording  on  the  feature:  color,  height,  width,  and  whether  it  needs  cleaning.  This  is  dependent 
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upon  the  actual  feature  itself.  Legend-  the  bottom  left  comer  of  the  map  that  either  explains  what  is  on  the  map,  or  allows 
you  to  choose  digital  instmctions. 

When  recording  measurements  for  manholes  such  as  diameter,  width,  or  height,  you  are  measuring  the  interior  (usable  portion)  of  the 
manhole.  Please  use  metric  measurements;  see  tape  measure.  Due  to  the  differing  shapes  of  the  manholes,  some  data  may  not  be 
applicable  (ie:  diameter  on  a  square  manhole,  width  and  height  on  a  round  manhole)  Such  as  follows: 


As  was  stated,  I  am  not  allowed  to  intervene  during  the  course  collection.  However,  if  I  note  that  you  have  made  three  of  the  same 
consecutive  errors,  I  will  intervene  and  we  will  continue  the  course.  It  will  be  annotated. 

You  will  be  timed  for  each  feature  collection.  Please  wait  until  the  RA  says,  -Start”,  and  then  let  them  know  when  you  are  finished  by 
saying,  -Stop”  for  each  feature. 

PLEASE  TALK  THROUGH  YOUR  FEATURE  COLLECTION  UNTIL  YOU  FEEL  THAT  YOU  ARE  COMPETENT. 

Scenario:  Talk  through  a  scenario  that  may  be  relevant  to  the  study  subject,  based  upon  their  demographic,  ie:  ROTC 
students,  the  use  of  the  digital  pen  and  software  in  a  tactical  environment;  Civilians:  the  use  of  the  digital  pen/software  in  an 
airport  or  by  paramedics. 

Your  method  order  will  be _ .  Explain  the  route  on  the  map.  There  are  50  points  (features)  on  the  course  to  collect.  They  are 

divided  by  13 (method _ ),  13 (method _ ),  and  the  remainder  to  40(method _ ).  The  last  10  will  be  your  preference  in  method.  You 

are  required  to  collect  2  per  method  of  existing  features. 

Do  you  have  questions  at  this  time?  ***RA:  Please  present  the  following  Methods  based  upon  the  Method 
Order  assigned. 


RA:  Check  Training  START  TIME 

Method  L 

Method  type  L  is  the  linking  ID.  You  will  have  a  map  and  a  spreadsheet.  Please  find  the  spreadsheet 
which  begins  with  the  column  — Ink  ID”.  So  you  will  orient  yourself  on  the  map. 

For  new  features:  You  will  touch  the  feature  in  the  legend  (manhole,  utility  pole,  address),  draw  it  on  the 
map  with  one  stroke.  This  drawing  is  for  you-  to  help  your  eye/mind  determine  what  you  are  putting 
where.  The  one-stroke  picture  is  your  choice:  a  circle,  a  line,  a  square.  Touch  link  to  form,  write  a  number 
on  the  map,  write  the  number  on  the  form,  and  then  fill  in  the  attribute  data. 
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Please  do  not  include  labels  as  they  are  already  built  in  (ie:  cm).  For  colors,  the  recognized  colors  are 
white,  yellow,  brown,  gray,  red,  black,  silver,  and  other.  For  utility  poles,  Kind  =  UNK/Unknown, 
TRF/Traffic/Traffic  Control/Signal,  TEL/Telephone/Telco/POTS,  ELE/Electric, 
COM/Combined/Combo/Multi,  BUS/Electric  Bus/Bus/Trolly/Train,  LAMP 

**For  existing  features,  two  methods  of  correction  are  available: 

1 .  Make  a  new  row,  but  use  the  same  linking  ID  as  the  existing  feature. 

2.  Write  the  new  data  in  the  same  row  as  the  existing  information;  do  not  change  the  linking  ID. 
Practice.  Questions? 

RA:  Check  Training  STOP  TIME 
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RA:  Check  Training  START  TIME 

Method  T 

Method  type  T  is  Temporal  Association.  You  will  have  a  map  and  a  spreadsheet.  Please  find  the 
spreadsheet  which  does  NOT  begin  with  — Ink  ID”.  So  you  will  orient  yourself  on  the  map. 

For  new  features: 

You  will  touch  the  feature  in  the  legend  (manhole,  utility  pole),  and  then  draw  it  on  the  map  with  one- 
stroke.  This  drawing  is  for  you-  to  help  your  eye/mind  determine  what  you  are  putting  where.  The  one- 
stroke  picture  is  your  choice:  a  circle,  a  line,  a  square.  Go  directly  to  filling  in  the  chart  with  the  attribute 
data.  The  time  stamp  on  the  map  and  the  time  stamp  on  the  spreadsheet  will  determine  which  data  goes 
together  (this  assigns  that  feature  a  row  on  the  chart). 

Please  do  not  include  labels  as  they  are  already  built  in  (ie:  cm).  For  colors,  the  recognized  colors  are 
white,  yellow,  brown,  gray,  red,  black,  silver,  and  other.  For  utility  poles,  Kind  =  UNK/Unknown, 

TRF /T raffic/T raffle  Control/Signal,  TEL/Telephone/Telco/POTS,  ELE/Electric, 
COM/Combined/Combo/Multi,  BUS/Electric  Bus/Bus/Trolly/Train 

**For  existing  features,  we  are  trying  to  determine  whether  the  current  attribute  data  is  correct.  For  this 
method,  touch  circle  feature,  then  circle  the  feature  on  the  map.  Write  the  attribute  data  on  the  spreadsheet 
in  a  new  row. 

Practice.  Questions? 

RA:  Check  Training  STOP  TIME 
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RA:  Check  Training  START  TIME 

Method  M 

Method  type  M  is  Writing  on  the  map.  You  will  have  a  map  only.  So  you  will  orient  yourself  on  the  map. 
For  new  features: 

You  will  touch  the  feature  in  the  legend  (manhole,  utility  pole,  address),  and  then  draw  it  on  the  map  with 
one-stroke.  This  drawing  is  for  you-  to  help  your  eye/mind  determine  what  you  are  putting  where.  Touch 
set  attribute  in  the  legend,  write  the  attribute  data  directly  on  the  map.  The  way  it  is  written  on  the  map  is 
in  the  x=y  format.  For  example, 

Color@  yellow 

Diameter@36 

Open@  true/  false,  yes/no 

Holes@0 

No  units  are  written  (ie:  cm).  For  attributes  that  would  be  checked  on  a  spreadsheet  for  their  presence  you 
would  write  true  or  false  (ie:  open  =  false;  clean=true).  For  colors,  the  recognized  colors  are  white,  yellow, 
brown,  gray,  and  other.  For  utility  poles,  Type  =  UNK/Unknown,  RET/Retail,  OFF/Office,  PAR/P arking, 
FOD/Food  Service/Food/Restaurant.  For  addresses,  business  type  =  UNK/Unknown,  TRF/Traffic/Traffic 
Control/Signal,  TEL/Telephone/Telco/POTS,  ELE/Electric,  COM/Combined/Combo/Multi,  BUS/Electric 
Bus/Bus/T  rolly/T  rain 

For  each  subsequent  attribute,  touch  set  attribute  first  and  then  write  it  on  the  map.  You  may  use  the 

attributes  listed  on  the  back  of  your  clipboard  to  determine  what  may  be  collected.  This  list  also  includes 
what  abbreviations  may  be  used  per  feature/attribute/data. 

FYI,  all  of  the  marks  that  you  make  on  the  various  map  sheets  will  be  downloaded  onto  one  cohesive  map 
sheet  with  an  attached  database  of  the  information  you  recorded.  On  the  computer  screen,  we  will  see  all  of 
the  ink  you  have  drawn,  and  the  assignment  of  the  features  on  the  map.  Then  we  will  be  able  to  call  up  a 
chart  of  the  attributes  as  the  program  has  read  your  writing,  so  it  is  important  to  write  neatly  and  in  the 
correct  orientation.  You  may  use  as  many  map  sheets  as  you  would  like. 

**For  existing  features,  touch  -e-ircle  feature”,  circle  the  feature  on  the  map,  touch  set  attribute,  write  in 
x@y  form. 

Questions?  Practice. 

RA:  Check  Training  STOP  TIME 


For  public  release,  distribution  unlimited. 


28 


Appendix  B:  Research  Assistant  timing  sheet 


SUBJECT  NO: 

INSIDE  TNG:  (M):  □  □  (L):  □  □(T):  □  □ 

TNG  Start: 

□ 

OUTSIDE  TNG:  (M):  □  □  (L):  □  D(T):  □  □ 

TNG  End: 

□ 

Totals  TNG: 

(M):  (L):  (T): 

METHOD  ORDER: 

Total  Complete  TNG: 

Course  started:  IZ 

Feature 

Method 

Start 

Stop 

TOTAL 

Coordinates 

Latitude 

Longitude 

1 

□ 

□ 

47°N36' 

122°W20' 

2 

□ 

□ 

47°N36' 

122°W20' 

3 

□ 

□ 

47°N36' 

122°W20' 

4 

□ 

□ 

47°N36' 

122°W20' 

5 

□ 

□ 

47°N36' 

122°W20' 

6 

□ 

□ 

47°N36' 

122°W20' 

7 

□ 

□ 

47°N36' 

122°W20' 

8 

□ 

□ 

47°N36' 

122°W20' 

9 

□ 

□ 

47°N36' 

122°W20' 

10 

□ 

□ 

47°N36' 

122°W20' 

11 

□ 

□ 

47°N36' 

122°W20' 

12 

□ 

□ 

47°N36' 

122°W20' 

13 

□ 

□ 

47°N36' 

122°W20' 

14 

□ 

□ 

47°N36' 

122°W20' 

15 

□ 

□ 

47°N36' 

122°W20' 

16 

□ 

□ 

47°N36' 

122°W20' 

17 

□ 

□ 

47°N36' 

122°W20' 

18 

□ 

□ 

47°N36' 

122°W20' 

19 

□ 

□ 

47°N36' 

122°W20' 

20 

□ 

□ 

47°N36' 

122°W20' 

21 

□ 

□ 

47°N36' 

122°W20' 

22 

□ 

□ 

47°N36' 

122°W20' 

23 

□ 

□ 

47°N36' 

122°W20' 

24 

□ 

□ 

47°N36' 

122°W20' 

25 

□ 

□ 

47°N36' 

122°W20' 

26 

□ 

□ 

47°N36' 

122°W20' 

27 

□ 

□ 

47°N36' 

122°W20' 

28 

□ 

□ 

47°N36' 

122°W20' 

29 

□ 

□ 

47°N36' 

122°W20' 

30 

□ 

□ 

47°N36' 

122°W20' 

31 

□ 

□ 

47°N36' 

122°W20' 

32 

□ 

□ 

47°N36' 

122°W20' 

33 

□ 

□ 

47°N36' 

122°W20' 

34 

□ 

□ 

47°N36' 

122°W20' 

35 

□ 

□ 

47°N36' 

122°W20' 

36 

□ 

□ 

47°N36' 

122°W20' 

37 

□ 

□ 

47°N36' 

122°W20' 

Notes: 
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SUBJECT  NO: 


Feature 


Method 


Start 


Stop 


TOTAL 


Coordinates 


Latitude 


Longitude 


38 


47°N36' 


122°W20' 


39 


47°N36' 


122°W20' 


40 


□ 


□ 


47°N36' 


122°W20' 


(User  pref.) 


41 


□ 


□ 


47°N36' 


122°W20' 


42 


47°N36' 


122°W20' 


43 


47°N36' 


122°W20' 


44 


47°N36' 


122°W20' 


45 


□ 


□ 


47°N36' 


122°W20' 


46 


□ 


□ 


47°N36' 


122°W20' 


47 


□ 


□ 


47°N36' 


122°W20' 


48 


47°N36' 


122°W20' 


49 


47°N36' 


122°W20' 


50 


47°N36' 


122°W20' 


Course  stop:  D 

Assistance  Requests: 
Method  (LJ,M): 


TOTAL  COURSE  TIME: 

□  □□□□□□  □□□ 

TOTALea.  Method: 


Today's  weather: 


User  Preferences 

Very 

Accurate 

Somewhat 

Accurate 

Neutra 

1 

Somewhat 

Inaccurate 

Very 

Inaccurate 

Q001 

1  prefer  method  M  for  its  organization. 

□ 

□ 

□ 

□ 

□ 

Q002 

1  prefer  method  M  for  its  simplicity. 

□ 

□ 

□ 

□ 

□ 

Q003 

1  prefer  method  M  for  its  neatness. 

□ 

□ 

□ 

□ 

□ 

Q004 

1  prefer  method  Mfor  its  timeliness. 

□ 

□ 

□ 

□ 

□ 

Q005 

1  prefer  method  M  overall. 

□ 

□ 

□ 

□ 

□ 

Q006 

1  prefer  method  Lfor  its  organization. 

□ 

□ 

□ 

□ 

□ 

Q007 

1  prefer  method  Lfor  its  simplicity. 

□ 

□ 

□ 

□ 

□ 

Q008 

1  prefer  method  Lfor  its  neatness. 

□ 

□ 

□ 

□ 

□ 

Q009 

1  prefer  method  Lfor  its  timeliness. 

□ 

□ 

□ 

□ 

□ 

Q010 

1  prefer  method  L  overall. 

□ 

□ 

□ 

□ 

□ 

Q011 

1  prefer  method  T for  its  organization. 

□ 

□ 

□ 

□ 

□ 

Q012 

1  prefer  method  T for  its  simplicity. 

□ 

□ 

□ 

□ 

□ 

Q013 

1  prefer  method  T for  its  neatness. 

□ 

□ 

□ 

□ 

□ 

Q014 

1  prefer  method  T for  its  timeliness. 

□ 

□ 

□ 

□ 

□ 

Q015 

1  prefer  method  T  overall. 

□ 

□ 

□ 

□ 

□ 

Notes:  (Bio  data):  year  in  school/ROTC,  program,  age,  previous  experience  with  technology/navigation,  1st  language 
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Appendix  C:  Interface  Workflow 

Protocol  for  continuity  --Soldier  as  a  Sensor  Attributes 

Before  Testing 

Open  Nucleus  Shortcut 

Folders 


•  e-folder:  Copy  JSfew  User4  folder  to  maintain  integrity  of  folder  ,  Rename  copy  to  =subject 
number4 

•  open  Excel  (Forms/Forms.xlsx)  and  Esri  Maps  (UrbanRecon_ACEl/UrbanReconACE.mxd) 
within  current  subject  e-folder 

•  Print  through  Capturx  toolbar  for  Capturx-enabled  sheets  (water-marked)  within  Excel  program 

•  Print  through  Capturx  icon  for  Esri  for  enabled  map  sheets 

Set  digital  pens 

Set  User  (subject)  number  on  Pen  Man  program  for  continuity  of  data  collection,  RA  pen= 
RA+Subject  number 


Post-testing 

Ensure  Nucleus  is  open. 

Insert  digital  pens  in  docking  station,  download  ink  from  both  into  computer.  Ink  will  be  located  per  file  in 
Desktop/Ink  Shortcut.  Click  to  process.  Cut,  paste  into  user's  file  under  Ink  folder. 

Open  user's  Excel  sheets.  Import  ink  into  cells  by  clicking  import  ink  button.  One  per  sheet  will  be 
necessary,  or  all  sheets  may  be  processed  per  user  at  the  same  time. 

Review  Excel  data,  comparing  to  paper  sheets.  Save. 

Import  map  sheets  data  by  pressing  =sessions  for  current  map4  radio  button.  While  still  in  edit  mode, 
review  link  ID  numbers  for  accuracy  in  attribute  tables  of  feature  classes.  Click  green  (check)  accept  radio 
button.  Import  Excel  data  into  the  attribute  tables  by  clicking  Jmport  cells  attribute  data4  radio  button. 
Validate  attribute  tables  for  time/accuracy,  using  corrected  excel  sheets  and  paper  maps. 
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